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IMPROVED METHOD AND APPARATUS FOR 
CREATING SOUNDS IN A VIRTUAL WORLD 

BACKGROUND OF THE INVENTION 
This invention relates to virtual reality systems 
and, more particularly, to a method and apparatus for 
creating sounds in a virtual world. 

Users of computer systems are now able to create 
virtual realities which they may view and interact with. 
One type of virtual reality system is disclosed in U.S. 
patent application No. 535,253, filed June 7, 1990, entitled 
"Virtual Reality Network," the disclosure of which is 
15 incorporated herein by reference. One task which must be 

performed is the creation of the virtual worlds within which 
the users interact. The virtual world should simulate the 
real world as closely as possible. Thus, not only must the 
animated world be created, but the sounds which one would 
expect to exist in the virtual world must also be provided. 



20 



SUMMARY OF THE INVENTION 
The present invention is directed to a method and 
apparatus for creating sounds in a virtual world. The 
25 system provides signal processing capabilities to convert 

monaural sounds to fully spacialized sound sources. A user 
of the system wearing a pair of stereo headphones perceives 
live, computer generated, or recorded sounds as coming from 
specific locations in space, just a listener does in the 
30 real world. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a block diagram of a particular 
embodiment of an apparatus according to the present 
35 invention for creating sounds in a virtual world. 
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BRIEF DESCRIPTION OF THE APPENDICES 
Appendix 1 is a text description of an apparatus 
according to the present invention for creating sounds in a 
virtual world; 

5 Appendix 2 is another text description of an 

apparatus for creating sound in a virtual world? and 

Appendix 3 is a source code listing for a program 
used for creating sounds in a virtual world. 

10 DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 

Fig. 1 is block diagram of an apparatus for 
creating sounds in a virtual world. A more detailed 
description of the apparatus shown in Fig. 1 appears in 
Appendix 2. The following describes some of the 

15 capabilities of the system. 

AudioSphere contains several innovations, including: 
l:Acoustic touch feedback using spatialized acoustic cues 
for "Grab/Hit/Unhit" 
20 2: Simulated and exaggerated Doppler shift cues using MIDI 
PitchBend; 

3: Parallel processing architecture, where rendering and 
other computations happen in a separate processor, connected 
to the host by a low-bandwidth channel 
25 Another item: MIDI-based generation of real-time sound 

effects in VR. This item is a prerequisite for 2, and a 
subsystem in our implementation of 1 and 3, but MIDI sound 
in VR as such may be too general and obvious a method for 
any specific patent claim. 

30 

1: Touch Feedback 

Touch feedback is a valuable element of computer/human 
interface, particularly when using the hand to grab 
simulated or "virtual" objects, as with hand-measuring 
35 devices like the VPL DataGlove. The present invention uses 
sound rather than tactile feedback to indicate correct 
gesture for grabbing objects (Grab) , actual contact with a 
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grabbable object (Hit) , and release of a previously Hit 
object (Unhit or Release) . In our implementation, the sound 
is three-dimensionally rendered and appears to come from the 
user's hand, but that need not be a requirement of the 
5 patent claim. Also, MIDI control of digitally sampled sound 
is our synthesis method, but that should not be a 
prerequisite of the claim. 



In our invention, sound feedback indicates several things: 

10 Grab: whether the current hand gesture allows the object to 
be picked up (Grab gesture) . In the current implementation 
a grab gesture results in a continuous sound that continues 
until the hand intersects with a grabbable object. We use a 
sound of continual suction sound, "sssss", to indicate the 

15 hand's potential for picking up an object. This suggests a 
"vacuum suction" model of picking up objects, rather than 
closure of the fingers around the object, and helps the user 
make a correct assumption about the user interface. 
Hit: whether the hand has intersected with the object to be 

20 picked up (Hit) object can be grabbed now. In the Virtual 
Reality system, motion of the object now follows motion of 
the hand. The Hit sound can be continuous until the object 
is released, but in the case of the vacuum suction model, 
the sound is "ssssp!" Another sound can continue while the 

25 object is being held, although in a system with other 
feedback (e.g. , graphics) this is not necessary. 
Unhit: whether the Grab gesture has ended and the currently 
held object has been released. If the vacuum suction model, 
we use a sound of reverse suction, again followed by 

30 silence: "Psssss." 



35 
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2: Doppler Shift 

In the physical world, Doppler shift is the increase or decrease of the pitch of a 
sound in accord with the speed of the object (rate of change of radial distance) 
relative to the listener. When a listener and object move toward eachother, the 
5 pitch of a sound emanating from the object goes up when heard by the listener. 
When they are moving away from eachother, the pitch goes down. The amount 
of pitch change is proportional to the fractional speed (rate of change of radial 
distance) of the objects relative to the speed of sound (about 600 miles per hour 
at common earth pressure and temperature). Thus the pitch of an object 
moving toward the listener at 60 mph is raised by about 10%. 

io AudioSphere, in conjunction with Body Electric and its DWs, generates Doppler 
shifts by raising and lowering the pitch using MIDI PitchBend capability built in 
to many modem music synthesizers. On synthesizers with polyphonic pitch 
bend capabilities, Bke the EMAX li synthesizer used in the current AudioSphere, 
several different sound sources can be doppler shifted at once. MIDI provides a 
low-bandwidth (typically 30 samples per second) method for the host computer 
and Body Electric to shift pitches of sounds emitted from objects in simulations, 

15 virtual reality, and other applications of AudioSphere. 

MIDI Is a hardware/software standard for generating and controlling sound in 
real-time on a low-bandwidth channel (31.25 Kbaud). MIDI PitchBend is a 14bit 
quantity that takes a range of values from O to 16,383. The lowest downward 
bend value is 0 and the highest pitch bend Is 16,383, with a middle value of 
8192 indicating no bend. 

20 

The Body Electric DMs allow the designer to specify the objects that have 
Doppler shifting, and to create attenuated or exaggerated doppler shifts as 
objects in the model move. The value for the PitchBend is determined by this 
formula: 

PitchBend « 8t92+(ScaleFactor * (Speed / SpeedOfSound) ) 
Speed is computed as the rate of change of radial distance between the object 

25 and the ear, using the GlobaiDistance DM in Body Electric. Speed is positive 
when the distance is Increasing, negative when the object moves toward the 
listener. The sign of the ScalePactor Is negative so when two objects are 
moving toward eachother, the PitchBend value goes up. The ScaleFactor can 
be adjusted depending on the specific PitchBend response of the MIDI 
synthesizer, which typically ranges for +-12% to 4-200%. The ScaleFactor or 
SpeedOfSound constants can be set to simulate very rapid motion, i.e. motion 

3 o over great distances with a correspondingly dramatic pitch shift due to doppler 
when the object passes by. 

Exaggerated doppler shift and exaggerated rolloff of sound loudness with 
distance may be useful claims in an AudioSphere patent Sound rolloff can be 
proportional to the distance, the distance squared, or any other exponent. The 
"cartoony" exaggerations heighten the VR or other user's perception of space 
3 5 and motion in the application. 
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? : £ *i a L ,el P roc *«*lng architecture 

SfSIT or m ? re /wfetera/ processors to compute sound 

«• 30 spa«al sound rendering modute(s), Kmlfng the amounl of 
computation that needs to be done on the central host running Body Electric 

?SS£fSSS? S n ™ ,BM - PC " 803W and 

387 math processor. Body Electric sends the peripheral processor cartesian 
coordinates relative to the head (from the center b^n tf^). Tte 
penphera processor performs conversion from cartesian (x. y. z) coordinates to 

sound spatiaiizer (in this case, the Convolvotron subsystem). 

S5?™°t!l! cartes ! an coordinates lets the peripheral processor 
E^ZhS ?^ 9,y ex P flns,VB trigonometry without taxing the hostprocessor 

system At^J^Sa^r'l^ s ^ lh0 reaMlma Parformarice of the 
system. At the same time, the head-relative cartesian representation Generated 

Al 0 !?*™" 1 DMs "! ^ E,ectrtc - **** «he SmpSSnlnX 
Peripheral processor, which does not need an ongoing model of the head's 



While the above is a complete description of a 
preferred embodiment of the present invention , various 
modifications may be employed. Consequently, the scope of 
25 the invention should not be limited except as described in 
the claims. 



WO 92/09921 



6 



PCT/US91/08947 



WHAT IS CLAIMED IS ; 

1. An apparatus for creating sounds in a virtual 
world comprising: 
5 coordinate means for providing cartesian 

coordinates of a sound-producing object located in a virtual 
world; 

transform means, coupled to the coordinate means, 
for transforming the cartesian coordinates to polar 
10 coordinates; and 

sound generating means, coupled to the transform 
means, for generating a sound which is perceived as 
originating from the cartesian coordinates in the virtual 
world. 
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